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1. Introduction 

Charles Stein has introduced a general method for proving limit theorems with 
explicit error terms, the essential idea of which is the notion of a characterizing 
operator. Given a probability measure P D on a space f2 with expectation E OJ a 
characterizing operator T is an operator on a suitable function space on f2 with 
the properties that: 

1. E T = 0. 

2. If E is expectation for a different probability on f2 and WT = 0, then 
E = E . 

The idea is then to prove that E = E by showing that ET = 0. To do this, 
Stein has introduced a method which he calls the method of exchangeable pairs. 

In this survey, we specialize his general approach to Poisson approximation. 
Very roughly, the idea is to show that a random variable W has an approx- 
imate Poisson distribution by studying how a small stochastic change affects 
the law of W. An appropriate such change is often easily found by constructing 
a reversible Markov chain resulting in an exchangeable pair (W, W). Stein's 
general approach is developed for the Poisson setting in section [21 This devel- 
opment is somewhat technical; as motivation for the machinery, we offer the 
following rough overview. The aim is to prove that W = X-i has an approx- 
imate Poisson distribution, where the Xj are (perhaps dependent) indicators. 
First, a probabilistic construction is used to create a second random variable 
W' such that (W, W) is exchangeable. This construction is often one step in a 
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reversible Markov chain with W as stationary distribution. Then the machinery 
from section |21 gives the following bound: 

\P(W € A) — Poi x (A)\ < (0.7)A- 1/2 (£i + E 2 ) 

with 

E 1 = E\\-cW(W' = W+l\{Xi})\ 
E 2 = ¥,\W - c¥{W' = W - l\{Xi})\ 

with c a parameter chosen to minimize the error terms. Thus the error will be 
small provided 

F{W' = W +1\{X,}) = ^ 

and 

^ W ' = W-l\{Xi}) = — . 

c 

In applications, W = X- L with the Xi indicators of rare events. The X$ 
are thus mostly zero and W of them are one. The example to keep in mind 
is when a point of the sequence is chosen at random, and that X, t is changed 
to its opposite, giving W'. Heuristically, since most of the Xi are 0, there are 
constants a and b such that 

¥{W' = W + l\{Xi}) = a 
P(W' = W- l\{Xi}) = bW. 

By the symmetry of the exchangeable pair, 

a = E[¥(W = W + l\{Xi})} 
= P(W' = W + 1) 
= V(W = W' + l) 
= E[¥[W' = W - l\{Xi})] 
= E(bW) 
= bX. 

Thus choosing c = — gives b = -, and makes the error terms small. In many 
cases, just changing one Xi to its opposite doesn't quite work to produce an 
exchangeable pair, but most of our exchangeable pairs are constructed in ways 
which arc rather similar to this. The reader will see rigorous versions of these 
heuristics starting in section |3J 

The contents of the rest of the paper are as follows. Section [21 develops a 
general bound. In section |3| we give a first example of using Stein's method: 
Poisson-binomial trials. The method is shown to give close to optimal results 
with explicit error bounds. In sections EE the method is applied to three clas- 
sical probability problems: the matching problem, the birthday problem, and 
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the coupon collector's problem. Sections and |H1 give multivariate and Poisson 
process approximations. 

The Poisson heuristic and the Chen-Stein approach to Poisson approximation 
have been actively developed since 1970. Substantial contributions from Louis 
Chen, Andrew Barbour and their many coauthors are surveyed in sectionsEland 
1101 These approaches share common roots with the present exchangeable pair 
approach, and all have different strengths and weaknesses; we attempt some 
comparison. Section ITT1 lists some interesting open problems. 

Acknowledgements. This paper is based on notes from a seminar at Stan- 
ford University in the winter of 2004. We thank the other speakers and par- 
ticipants, particularly Susan Holmes and Charles Stein. We also thank Geir 
Helleloid for a careful reading of the manuscript. 



2. The Method of Exchangeable Pairs and Poisson Approximation 

Let W be an integer-valued random variable defined on a probability space 
(fl,A,¥). Let X be the bounded measurable functions on f2. We write E/ = 

J f(u))f(du) and think of E as a linear map, E : X — * R. Throughout we 

suppose that E(VF) = A < oo. Let X D be the bounded real-valued functions 
on N = {0, 1, 2, ...}, and let E D : X D — ► M be expectation with respect to Poi\ 
measure on N. The random variable W allows us to define 

by 

/3/H = (1) 

Heuristically, W has an approximate Poi\ distribution if E G (/) = E(f(W)); i.e., 
E = E/3. This is equivalent to saying that the following diagram approximately 
commutes: 




Stein constructs a symmetric probability Q on S7 x 17 with margins P (i.e., 
Q(A, B) = Q(B, A) and <Q(A, ft) = P(A)), which gives an exchangeable pair. It 
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is used to define the following enlarged diagram: 

J - -X (3) 




3" o ^ ^ X G 

Stein's lemma, developed and proved in lemma J2J below, shows in a precise 
sense that if the left square approximately commutes, then the triangle approx- 
imately commutes. This leads to explicit bounds on Poisson approximation. 

We begin by constructing the top row of the diagram. We define the charac- 
terizing or Stein operator T for the Poi\ distribution by 

Tof(j)=Xf(j + l)-jf(j) (4) 

and we let 3^ C X be the functions / : N — > R such that T a f is a bounded 
function. Note that £F contains all of the functions / on N such that f(n) = 
eventually, thus it is rich enough for our purposes. 

A simple calculation verifies that E T = 0; i.e., if X ~ Poi\, then 

E(Xf(X + l)-Xf(X)) = 

for every bounded function /. 

Remark: T is called characterizing because it also has the property that if 

oo 

p is a probability on N with the property that /~^(A/(j + 1) — jf(J))p(J) = 

3=0 

for every bounded function / : N — > R, then p is the Poisson distribution with 
parameter A. To see this, let / = Sk- This yields the equation Ap(k— 1) = kp(k), 
which leads to a recursion relation that describes Poi\. 



Next we will define a map U , which is almost an inverse to T . Define: 

UofU) = E^(/W - Eo/). (5) 

k=Q 

It is easy to check that 

ToUofij) = f(j) - Eof. (6) 

Thus U is inverse to T Q on fcer(E ). The following lemma, proved by Barbour 
and Eagleson in 0, gives bounds on expressions involving U . 
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Lemma 1. 1. For f e X with < / < 1 and j £ N, the map U of 0) 

satisfies 

\U„M\ < min(l, 1.4A- 1 / 2 ) 

\u f(j + i)-u f(j)\ < 



A 

f i o = n 

2. Lei <5 (j) 



i i = o, 

otherwise. 
For f(j) = d](j) - e-\ 



|C/o/(j)l < 



A reviewer has pointed to a sharper bound in [HE]- We have not seen this 
and will use lemma (QJ in what follows. This lemma and equation © show that 
U f G Jo for every / £ 1„. To complete the top row, let i denote the map 
K — > X which associates to each constant c the map on N which takes the 
value c for each n. 

To construct the bottom row of diagram 10), define 

1 = {/ : x £1 — > E : / bounded, measurable, and/(cj,a/) = — f(u)',u>)}. 
Using a probability Q on 57 x 17 as discussed above, define the operator T by: 

77H=E Q (/( W ,u/)H. (7) 
Observe that T/ is a bounded function on 17 for / G 3\ Further, 

ET/ = J f(u,ui')Q(du,dui') 

= J f{u',oj)Q{duo,du') 

= - J f(LU,Lu')Q(duj,dl0') 

where the first equality is by the symmetry of Q and the second is by the 
anti-symmetry of /. Thus ET = 0. 

Finally, define a : 3^ — > J to be any linear map, for example, af(u),u)') = 
f(W(ui)) — f(W(u)')). Stein's lemma is true regardless of what a is, so we choose 
a to work well with the problem at hand. In applications, a is often a localized 
version of the example given above; see proposition J3J below for an example. 

We can now state and prove: 



S. Chatterjee et al. /Exchangeable pairs and Poisson approximation 



Lemma 2 (Stein). Suppose that in the following diagram of linear spaces and 
linear maps, ET = and T U = Id — iE Q . 




(8) 



Then 



Proof. We have: 



E/3 - E = E(/3T - Ta)U Q 



ETaU 

E(Ta-f3T )U + Ef3T U 
E(Ta - (3T )U + E/3(Id - iE Q ) 
E(Ta - /3T )U + E(3 - E 



(9) 



as desired. 
Remarks: 



□ 



1. 



2. 



Stein's lemma makes precise the sense in which the Poisson approximation 
E/3 = E Q holds provided (3T — Ta is small; this expression must have small 
expectation on the range of U . A good choice of a can put this expression 
into a useful form (see proposition below). 

In what follows, the equality in Stein's lemma is used to bound the error 
term E(Ta — [3T )U with analysis. The same equality can also be used to 
simulate the error term using the exchangeable pair as a reversible Markov 
chain. This approach is developed in Stein et al. (2003). 
Stein's method is often used to study a random variable W which is a sum 
of indicators Xi. In the previous development, the operator T was defined 
as conditional on W . The lemma holds if T is defined by conditioning on 
any cr-algebra with respect to which W is measurable; in particular, one 
can condition on {^}. An application of Jensen's inequality shows that 
conditioning on W alone rather than conditioning on {X{\ always gives 
better results. However, conditioning on only W can make bounding the 
error terms more difficult, and in practice we often condition on all of the 

One reason for presenting the diagrammatic view of the subject is that it 
applies, in essentially the same way, to any type of approximation. The 
fact that we're approximating by a Poisson random variable enters into 
the diagram only in the top row, as the characterizing operator is the one 
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corresponding to the Poi\ distribution. To use Stein's method in other 
contexts, one needs only to find the appropriate characterizing operator; 
the rest of the picture remains unchanged. Holmes (in chap. 3 of [23] ) gives 
bounds for general discrete measures. 
5. Note that for lemma J5J) and the bounds to follow, it is only required that 
W and W' are exchangeable. While this is most easily achieved by making 
Q symmetric, examples of Fulman and Rinott-Rotar 03] show that 
other constructions are possible. 



The following proposition gives us a more workable form for the error term. 

Proposition 3. Let W be a random variable on (il,A, P) and Q a symmetric 
probability on O x Q with margins P. Let W = W{u>) and W = W{uj'). Then 

1. 0T o f = Xf(W + 1) - Wf(W) for f e X . 

2. Let af{uj,uj') = cf(W')I {w >- 1} (W) - cf(W)I {w _ 1} {W) for a constant 
c. Then for / 6 J„, 

Taf(cu) = cf(W+l)Q(W = W+l\uj)-cf(W)Q(W' = W-l\(j). (10) 

Remarks: For this choice of a, the error term from Stein's lemma will be 
small if the exchangeable pair can be chosen so that 

W 

Q(W = W - l\w) sa — (11) 

c 

Q(W' = W + 1\lu) « ^ (12) 

for some c. One then uses this c in defining a to obtain cancellation in the error 
term, and then uses the bounds in lemma (JTJ to bound the error term. The 
examples which follow show that i|ll|) and (|12f> often hold for natural choices of 
Q, for some c. We observe that if l|ll(l and (|12f> hold, the ratio method proposed 
by Stein is also a way to approach Poisson approximation. 

3. Poisson-Binomial Trials 

Let Xt, X2, X n be independent {0, 1}- valued random variables with P(Xj = 

n 

1) = pi and PpQ = 0) = 1 — pi, such that / ^Pi = A. To put this example into 

i=l 

n 

the framework of section 101, let = {0, 1}", with F(u u ...,u n ) = J^p"^ 1 ~ 

Pi) 1 ^ UJi 1 and let X be the bounded functions on fi. Let W(lo) = Yli=i Ui - *-* ne 
way to build a symmetric probability Q on f2 x f2 is the following probabilistic 
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construction: choose an index / uniformly in {1, ...,n}, and let ej = 1 with 
probability pi and ej = with probability 1 — p/. Given w e fi, set lo[ — 

{lo if i 9^ ./^ 
1 ' . This constructs a probability Q on f! X !1: assign to the pair 

tx if i = I- 

(u>,u>') the probability of choosing u> from f2 according to P and then going from 
uo to uo' by the process described. It is clear from the construction that Q is 
symmetric and has margins P. From these definitions, 

1 " 

= W - = -VwiCl-pi) 

»=i 



1 - 

Q(W = W + l|w) - -V(l- Wi ) ft 

n 

i=l 



n 

Combining these calculations with (fTTTf) and choosing c — n gives 

n n 

{pr a -Ta)f{u>) = [\f(W+l)-Wf(W)]-[f(W + l)(\-J2Pi"i)-f(W)(W-J2i 
= (/(W + l)-/(W))^Pi«i. 

i=i 

As we consider / = U g for functions g with < g < 1, lemma yields the 
following: 

Theorem 4. Lei Ai,AT2, AT„ be independent {0, l}-m/Med random variables 

n n 

with P{Xi = 1) = pi and ^p t . = A. Then, if W = X i? 

i-l i=l 

/I - p- x \ " 
||£.(W)-P«A||rv< 5>i 



2 

1/ 

i=l 



Remark: The factor of 2 in the denominator arises because 
MM — HI = sup|^(A) - = ^sup 

A ^ / 

where the first supremum is taken over all measurable sets A and the second is 
over all measurable funtions / such that ||/||oo < 1- 

Example 1: Let p% = For A fixed and n > A, the bound becomes ^— f- — ■ 
This bound was derived by a different argument by Barbour and Hall. A com- 
parison with other available bounds and approximations for the i.i.d. case is in 
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Kennedy and Quine |85| . where there are also extensive references given. The 
bound that we give is sharp for small A and can be improved for large A. 



Example 2: Letpi = i for 1 < i < n. Thus A = logn— 7+O (i). The bound 

/ (i_ £W 2 \ , , . . . 

becomes - | ( 1 + O ( — | | . Here, the factor of A in the denominator 

\ 12 log n 

saves the day. 




Remarks: Under the conditions of theorem (0}, there are methods for com- 
puting the exact distribution of W and a host of approximations that differ from 
the Poisson. See Stein or Percus and Percus 02] for further discussion. An 
extensive collection of random variables which can be represented as sums of 
independent binary indicators is in Pitman |43| . 



4. The Matching Problem 



Because of its appearance in Montmort JSJi the matching problem is one of 
the oldest problems in probability. It asks for the distribution of the number of 
fixed points in a random permutation. Takacs |53| gives an extensive history. 
To put this problem into our set up, let S n be all n! permutations of n objects 
with P(cr) = -h, and let W(a) = |{i : a(i) = i}\; that is, W(a) is the number of 

n 

fixed points of a. Note that W — Xi where Xi indicates whether a fixes i. 

i=l 

This approach of writing a random variable as a sum of {0, l}-valued random 
variables, called the method of indicators, is one of our main tools for computing 
expectations of N-valued random variables. Using this representation of W, it is 
easy to see that EW — 1. Build an exchangeable pair (a, a') by choosing a E S n 
according to P and then choosing a' given a by following a with a random 
transposition r chosen uniformly among the non-trivial transpositions in S n . 
Then 

= W -l\a) = 



W + l\a) = 



n(n — 1) 
2(n - W(a) - 2a 2 (a)) 
n(n — 1) 



where 0,2 (er) is the number of transpositions which occur when a is written as 
a product of disjoint cycles. The first expression is calculated by observing that 
if W = W — 1, then t must choose one of the fixed points and switch it with 
something which is not a fixed point. The second calculation is done by similar 
considerations. 

Define a to be the localized operator as in proposition and choose c = . 
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Then, for g : N -> [0, 1], 

(fiT - Ta)U o9 (a) =[U o9 (W + 1) - W ■ U o9 {W)] 



--U„g(W + i)2il±^2l _ U p(W)— 
n n 



(13) 



Using the method of indicators, it is easy to check that E(W^ 2 ) = 2 and E(2a2) = 
1. Further, for ||<7||oo < 1> ll^offllco < 1 by lemma |QJ. Putting all of this together 
proves 

Theorem 5. Let W be the number of fixed points in a randomly chosen per- 
mutation on n letters. Then 



mW)-Poi 1 \\ TV < 



Remarks: 

1. In this example, the bound is not sharp. As is well known, — 
PoziHtv < =t. It is interesting to speculate on just where sharp control 
is lost. We have proceeded by equality through $F$ . 

2. The super exponential bound of remark (1) is an algebraic accident. This 

can be seen by considering the number of fixed points in only part of the 

permutation, that is, considering Wfe(cr) = |{1 < i < k : a(i) = i}\ for 

some k < n. Essentially the same argument shows that Wk~ Poik. In 

section El we show that the fixed point process (scaled to [0,1]) converges 

to a Poisson process of rate 1. Here, the error may be seen to be of order 
1 

it ' 

Generalized matching 

Consider now the more general problem of fixed points of permutations of the 
set A = {1, 1, 1, 2, 2, k, k} where the number i appears U times. For 
example, if two ordinary decks of cards are shuffled, placed on a table and turned 
up simultaneously one card at a time, a match is counted if two cards of the 
same number appear at the same time, without regard to suits. This is the 
matching problem with k — 13 and U — 4 for each i. 

Let \A\ = n and let S n be the set of all permutations of the elements of A 
with P(er) = for each o. Let W(a) be the number of fixed points of a and 

k 

Wi(a) the number of times that i is a fixed point of er, thus W = \_. Wi- Let 

l 
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Wij(a) = \{i : a(i) = j}\, that is, Wij is the number of times a sends i to j. 
Finally, let 

1 if a(i m ) = j, 



X m (j) = 



otherwise, 



where i m is the mth i in the set A. 

Build an exchangeable pair in this situation in the same way as in the pre- 
vious situation: follow a permutation a by a random transposition of the n set 
elements. Then 



Q(W' = W+l\a) = * ]T]T^-^-^ 

\2J i—i jjii 
k 



t y 

n(n - 1) 



ll-2l i W i + Wi-J2 w ji w ij 



Q(W' = W-1\<j) = l^^n-^-Jj + ^i) 

nW-J2 l iWi - W 2 + Yj W i 



n(n — 1) 



i=l 



To calculate the first expression, make use of the fact that for fixed i, J2j^i Wj» 
k — Wi. The first line follows as one determines the probability in question by, 
for each fixed i and each j ' ^ i, counting the number of ways to choose a symbol 
i with <j{j) = i and a symbol k ^ i, k 7^ j with a(i) = k. These will then 
be switched by a transposition, and the number of fixed points will have been 
increased by 1. The second calculation is done similarly. 

In order to bound the error term, the following moments arc needed: 



(k h 
EE*™ 1 
i—l rn — 1 



e(w 1 ) = e(j2x: 11 ( 1 )\ =|. 

\m=l / 



E(Wf)=E (J2 X m (i) 



\rn—l 



lf[n + l 2 -2h} 
n(n — 1) 
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h h 
\.p—l m— 1 



i 2 i 2 



n(n — 1) 



(for ij^j). 



E ( W 2 - E w i 



E [EEEE«w 

. i=l p=l m— 1 



Now choose (in analogy with the previous case) c = 1L -^- and make use of 



proposition Q to estimate: 



|E/(W + 1)[A 



< (1.4)A- x / a lE 



' = l|cr) 



E 



2^- Wf + E^yWj* 



< 



1.4 



fe \ / fc \ / fc 

e(Ew]+ e IE^ 2 ]+ e (EE^ 



1.4 



>(n-l)AV3^ 



E (2n-4)Zf + i?Ef 



1.4 



n 2 (n-l)AV2^ 

" / a 3 /2 

< (1-4) ^-r 
\n — 1 

for / = U Q g with g : [0,1]. 



E[(2n-4)Zf+^(nA)] 
2 
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For the other half of the error term, 



\Ef(W)(cQ(W' = W-l\a)- W)\ < 



1.4 



1.4 



n 2 (n - 1)A 



lKn-l) + Y^m 



1)AV2 ^ 



i=i 



< (1.4) 



A 3 / 2 

71—1 



\l/2 ) 



i 2 A^ 



iox f = U o9 andg :N^ [0,1]. 

Putting these two estimates together with proposition © proves 

Theorem 6. Let A = {1, 1, 2, 2, fc, fc}, imf/i the number i appearing 
Li times, \A\ = n. Let W be the number of fixed points of a random permutation 

of the elements of A, and let a — if. Then A := E(W) = — if and 

^— ' n ' 

i=i 

A 3 / 2 3/i 



i=l 



||XL(W0 - PoixWrv <(1.4) 



1 2ti 2 A 1 / 2 



Remarks: 

1. In particular, if Z j = i for each i, then the theorem gives: 

71—1 

2. A celebrated paper of Kaplansky 33 first showed that W had an approx- 
imate Poisson distribution under suitable restrictions on the U. 

3. Stein's method may be used to prove Poisson approximation in the match- 
ing problem for some non-uniform distributions /i on S n . In outline, the 
technique is similar: construct Q on S n x S n by choosing a according to 
fi and then choosing a' by making a random transposition and using the 
Metropolis algorithm to ensure that a' is also /^-distributed. 

4. Essentially the same techniques can be used to study approximate matches. 
Here, W = |{z : |z — o(%)\ < c}\ for some c. 

5. Generalizing the problem in remark 4, one may study A i7r (i) where 
Aij is an n x n matrix with entries in {0, 1}. Similarly, one may study the 
number of permutations which violate a given set of restrictions at most 
fc times with this method, if the restrictions are given by a {0, l}-matrix 
(that is, ir(i) = j is allowed if Aij = 1). Classically, these problems are 
solved using rook theory or the extensive development connected to the 
permanent (see |21j for extensive references). 
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5. The Birthday Problem 

For the classical birthday problem, k balls are dropped independently into n 
boxes, such that any ball is equally likely to fall into any of the n boxes. How 
many balls should there be to have some specified chance of having a box with 
more than one ball in it? Let us set up the problem in the framework of Stein's 
method. Let [n] = {1, . . . , n}, and let Q — [n] k . An element u) = (&i, . . . ,bk) € 
will represent an arrangement of the k balls in the n boxes: bi is the number of 
the box into which we put ball i. Thus the bi are i.i.d. and uniformly chosen from 
[n] . Denote this probability measure on Q by P. Let M m be the number of boxes 
containing exactly m balls, and let M m+ be the number of boxes containing m 
or more balls. We will show that if k = then W = M 2 + is approximately 

Poisson distributed with parameter ^- . 

Consider the random variables M 2 , W and V = l{h = bj}. Clearly, M 2 < 

W < V. Also, they are all equal when M 3+ = 0. Now M 3+ < ^ I{6j = b 3 = 

i<j<i 

bi}, since any box which contains more than two balls must contain one such 
triplet as in the sum. Thus, 

E(M 3+ ) < Yl nh = bj = h) = (3) ^ < 

i<j<l \ ' 



6-v/n' 



e 3 

Thus, P(M 3+ ^ 0) = P(M 3+ > 1) < — 1=. In particular, d TV {^w^v) < 

g3 /u\ 1 (p. 

P(W^V)< — =. Note that EV" = - w — . 

6y/n \2 J n 2 

Build an exchangeable pair (u),u)') as follows: choose w £ O according to 

P and then choose an index I e [k] uniformly. Let b} be i.i.d. with the {bi}, 

and let u>' — (61, . . . , b}, . . . , bk)- Thus the exchangeable pair is formed by first 

distributing the k balls independently and uniformly in the n boxes, and then 

choosing a ball at random and choosing a new box for it at random. As before, 

let W = W(lo), W = W(J), and compute: 

2M 2 (n- Mi - 1) 
kn 

Mi /Mi 1\ k- Mi - 2M 2 /Mi 



r ' = W-l\u)) = 
Q(W' = W + = 



k \ n n J k \ n 

M 1 (k~2M 2 - 1) 
kn 



The first equality holds as in order to make W go down by 1, we must choose 
a ball from a box that contains exactly two balls and put it into any box except 
cither the one it came from or any of the boxes which started with exactly one 
ball. The second computation is similar. 
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By proposition J3J and choosing c = |, the error term for Poisson approxi- 
mation is given by: 

\ snp{E(Af(W + 1) - Bf(W)) : / : N — > [0, 1]} 



where 



and 



A 



B = W 



I 2 Mi(fc-2M 2 -1) 
2 ' 2n 

M 2 (n-Mi - 1) 



Observe that 9 ~ kn 1 / 2 implies that 



_ fc(fc-Mi) + 2M 1 M 2 +Mi 



2n 

32 



Now E(fc - Mi) = E(VT) < E(V) < 9*/2, and E(MiM 2 ) < fcE(M 2 ) < k9 2 /2. 
Thus, 



E\A\<^r-+9 2 + l 



2n V 2 



1 



Finally, 
EIBI = E 



W -M 2 



M 2 (Mi + 1) 



^ + 3^ + 2^ 



since E(M3 + ) < as shown before. Applying lemma Q), proves 

Theorem 7. // we drop k = 9y/n balls independently and uniformly into n 
boxes, and let W be the number of boxes containing at least two balls, then 



\L(W) -Poig2 /2 \\ TV < min{l, V26T 1 } 



196» 3 + 69 9 2 
YlJn + 2n~ 



Triple matches 

We next consider a variation of the birthday problem: triple matches. As before, 
let fi be the space of possible arrangements of k balls in n boxes with proba- 
bility P given by independent and uniform placement of the balls. This time, 
the random variable W{uj) will denote the number of triple matches in lu; i.e., 
\{(i,j,l) : I < i < j < I < k,bi = bj = bi}\, where bi is the box in which we 
put ball i. One can write W = ^{b i= bj=bi}i and from this representation, 

Kj<l 

we get EW = 



\3J n 2 
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Let Mi(uj) be the number of boxes containing exactly i balls, thus W = 
f J Mi. Let X\ be the indicator that box i contains exactly I balls. Then 



E(Mi) 



it-; 



Construct an exchangeable pair in this case exactly as in the previous case: 
choose a ball at random and then choose a new box at random to put it in. We 
have: 

= W + l\u) = ^- [MiM 2 + 2M 2 {M 2 - 1)] 



— \M 1 M 2 + 2M. 
kn 



q 



2M 2 



This follows as the only ways to make the number of triples sharing a box go up 
by exactly one are either to choose a ball from a box with only one ball and move 
it to any of the boxes with two balls, or to choose a ball from a box with two balls 
and put it into one of the other boxes with two balls. No further possibilities 
exist, as removing a ball from a box with m balls and putting it into a box with 
I balls will increase W by exactly one if and only if (™) — ("V 1 ) = (2) — an< ^ 
there are no solutions of this if m > 3. Similarly: 



= W-l\u) = — [3M 3 M + 3M 3 Mi]. 
kn 

Using lemma Q and proposition J2J together with the triangle inequality, 
we will need to estimate: 



E 



ft(jfe-l)(fc-2) 



6n 2 



kn 



[MiM 2 + 2M 2 2 - 2M 2 ] 



E 



— [3M M 3 + 3MiM 3 ] - W 
kn 



(14) 
(15) 



for some choice of the parameter c. 

Consider l|14|) first. Define new random variables M, = Mi — \ii where fXi — 
EMj and write everything in terms of these Mj. We have: 

MiM 2 + 2M%-2M 2 = MiM 2 + 2M 2 2 + (^ 1 +4/i2-2)A72 + Ai 2 Mi+/ii/i2-2^2 + 2^ 
The deterministic part of this is 

k(k - l)(n - l) k - 2 [k{n - if- 1 - 2n k - 1 + k(k - l)(n - if- 2 ] 



[ii(i 2 - 2/i 2 + 2fi 2 = 



2n 2k ~ 2 



S. Chatterjee et al. /Exchangeable pairs and Poisson approximation 



80 



and if k = o(n), then the top order term of the numerator is k(k — l)(fc— 2)n 2k ~ 3 . 
We thus choose the parameter c to be c = -|. Then in expression 1)140. use the 
triangle inequality to estimate the deterministic part of the sum separately. It 
is asymptotic to ^j, thus if k = o(n 3 / 4 ), this part of (fH|) goes to 0. Recall that 
EW = fc ( fc ~^ fc ~ 2 ) ; go to get a nontrivial limiting distribution, the case we are 
interested in is k = 9n 2 ^ 3 for some fixed 6, so limiting our considerations in a 
way that still allows for this case is no loss. We will now estimate each term 
of the non-deterministic part of (fH|) separately. Using the method of indicators 
gives 

E(M 1 2 ) = E(M?) - n\ 

k(n - l>*2^(ri - If- 2 + (fc - l)(n - 2) fc ~ 2 ] - k 2 (n - l) 2fc ~ 2 

^ = 0(k). 
E(M|) = E(M 2 )~^ 2 

k(k - l)(n - l)n k - l [2{n - l) k ~ 3 + {k-2)(k- 3)(n - 2) fe - 4 ] - fc 2 (fc - l) 2 (n - l) 2fc - 4 



4n 



2fc-2 



Now to estimate (|14H . use Cauchy-Schwartz: 

fc(fc- l)(fc-2) 1 r , 
E -i r4 [MiM 2 + 2M 2 2 - 2Af ; 



, | . .., _ _^.^2j 

on z An 

< o | + JLKiSwft, + Aea7| + a+^nfti + £e|a7,| 



n 3 / \ n 3 / 2 / \n 2 I \ n 3 / 2 



In particular, if fc = 6n 2 / 3 , this expression goes to as n — > oo. 

The same kind of analysis is used to bound Q15p. Again, recentering all the 
random variables about their means and using our choice of c, the expression 
<|15|) turns into: 

l \TfT~r> Tf , m^mma TF , j7i /At? , + fc(fc-l)(fc-2) 

£=3 v 7 

(16) 

The deterministic part approaches 0: 

Mo/^3 + Mi/i3 fc(fc-l)(fc-2) (n-l) 2fe - 4 fc(fc-l)(fc-2)(n-l + fc) fc(fc-l)(fc-2) 

n 6n 2 6n 2fe_1 6n 2 

'fc 4 ~ 
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In order to estimate the rest, the following moments are needed: 



E(M 2 ) 



E(M 2 ) - fj,Q 

(n - l) k n k - 1 + {n- 2) k (n - l)n 



(n - 1) 



2k 



= 0(n). 

E(M 2 ) = E(Af 2 ) - [i\ 

n k - x [k{n - \) k - x + k(k - l)(n- l)(n - 2) k - 2 } ~ k 2 (n - 1) 



E(M 2 ) = E(Af|) — jU 2 , 

fc(fc - l)(n - l)n fc - 1 [2(n - l) fc - 3 + (fc - 2)(k - 3)(n - 2) fc - 4 ] - fc 2 (fc - l) 2 (n - 1) 




E(M 2 ) 



E(M|) - ^ 2 



rt fc ~ 1 fc(fc-l)(fc-2)[6(rt-l) fc ~ 3 + (fc-3)(fc-4)(fc-5)(n-l)(K-2)' c ~ 6 ]-fc 2 (fc-l) 2 (fc-2) 2 (rt-l) 

36n 2fc - 2 




E(M 2 ) 



E(Af 2 ) - M 2 



n k-i [{n _ l)k -,^ q { k-, ){n _ 1)(n _ 2)fc _ 2<] _ 1} 




E(M 2 ) 



E(M 2 ) - M 2 
(n — l) k ~ e n 




(for | < 4 < k). 



Looking back at and using the triangle inequality and Cauchy-Schwarz 
on most of the terms, the following go to zero: E\±M 3 M \, E|^ 3 M |, E\^MxM 3 \, 

E|i/i3Mi|, and E|i/ziM3|. We have already dealt with the deterministic part. 
Next, consider the term 
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which is the remaining ^iqM^ term of (|16[) and the first summand of ( J Mg. 



£=3 



E 



1 



-HqMz - M 3 



n 



E 







(n-l) k 



fc 3 / 2 



1 Af? 



fc4 



Finally, we have to deal with E^^ j ) \ Mg\. Treat this expression in two parts. 



No matter what n is, estimate the first 24 summands by ( J • O 



27 



,3/2 



The 



rest is bounded as follows: 

A- 



E 

£=28 



\MA k-k 6 -0 



O 



.,14 



,27/2 



,27/2 



so if k = o (n 3 / 4 ) as above, then this goes to 0. This proves: 

Theorem 8. Let k balls be dropped independently and uniformly into n boxes, 
where W is the number of triples of balls sharing a box. Then, 



-o{* 



TV 



In particular, if k = 9n 2 / 3 , then W has a non-trivial limiting Poisson distribu- 
tion. 



Remark: It is instructive to compare the bound in theorem |H1 to the more 
general bound for multiple matches in the birthday problem that appears in 
theorem 6B of the book of Barbour, Hoist, and Janson. First, their bound gives 

f k 4 \ 

an explicit inequality instead of the O I ] of thcorem|SJ More importantly, in 

\n V 

the critical case where k = 8n 2 / 3 , theorem|Hlgives a bound of order n" 1 / 3 for the 
error. Theorem 6B gives a bound of order n -2 / 3 . This suggests a more careful 
look at the analysis above and a healthy respect for the coupling approach. 

6. The Coupon-Collector's Problem 

In its simplest version, the coupon-collector's problem is as follows: drop k balls 
independently and uniformly into n boxes. How large should k be so that there 



S. Chatterjee et al. /Exchangeable pairs and Poisson approximation 



83 



is a prescribed chance (e.g. 95%) that every box contains at least one ball? This 
is also an old problem: Laplace (1780) gave an effective solution. Feller 120] and 
David-Barton [T?| present substantial developments. 

Write W for the number of empty boxes, and use 9 to write k = n log n + 9n. 
We introduce the new notation ki for the number of balls in box i, and use Ni to 
denote the number of boxes with i balls. Note that E(VF) = n(l ~ n~ 1 ) k ~ e~ e . 
Hence we take A = e~ e . Make an exchangeable pair by choosing a ball at 
random, choosing a box at random, and putting the chosen ball into the chosen 
box. As before, compute: 

kn 

F(W' = W + 1\.) = N ^- W -V 

kn 

Taking c = n gives the error for Poisson approximation as 

sup{E(A/(W / + 1) - Bf{W)) : < / < 1} 

where 

A = er e_ N^n-W-l) 
k 

and 

B = w (k-N 1 )W _N 1 W_ 
k k 

By the method of indicators one can compute ENiW = Yltej ^{^i = lAfcj = 0), 
where ki is the number of balls in box i. Hence, 

1 , Jk\l f 2 N k ' 



< n(l-2n- 1 ) k ~ 1 

p -28 

< . 

~ n- 2 

Now E\A\ < E\C\ + E\D\, where 

_ e (e- e \ogn)(n-W-l) 
C - E k 

(e- 9 logn)(n - W - I) N x {n-W - 1) 
k k ' 



and 

D 



We have 

_ e- e (n\ogn + 9n ~n\ogn + (W + l)logn) _ e~ e (9n + (W + l)logn) 

^ — ', — ', 
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~{8n+ (e- 9 + l)logn) 6e- e 
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E|C| < 



n log n + On 



logn ' 



71 loff ?2 

So what remains is to bound E|D|. Observe that E|£>| < — E 



e — 



logn 



E 



Ni 
logn 



— e 



n + 



On 
logn 



1 

1 - - 

n 



k-l 



< 



< 



nOe 



(n — 1) logn 
ni 



n|l 

n 



— e 



k-l 



i0e~ 9 



(n — 1) logn n — 1 
where a — logn + fclog(l — n _1 ). Now 

1 



a = logn + (n logn + On) log ( 1 
= log n + (n log n + On) 
= —0+(n\ogn + 0n) 



n 

I 1 1 

~n ~ 2^? ~ 3^3 

1 1 

2^ ~ 3^3 



In particular, a < — 6*. Hence 

|e Q -e- e | < e max{a '- e} \a- (-0)\ 
< e~ e (n log n + On) 
e~ 6 {\ogn + 0) 



1 1 

2^2 + 3^3 



< 



2(n- 1) 



So if concentrates at its expectation, there is a bound on the error term. 

logn 

This is seen as follows: 



E 



E 



log n \ log n 



< 



VVar(JVi) 
logn 



Now iVi = Y^i=i £*> w bere & = = 1}, fcj being the number of balls in box 
i. So 

Var(iVi) = nVar(a) + n(n - l)Cov(£i, 

Let 



P = p(a = i) = 



fc\ 1 
1/ n 



1 



1 



fe-i 



~ e logn 
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and 



p = P(Ci = 1 A 6 = 1) = k(k - 1)4 ( 1 - - 



fc-2 



Then Varf^) = np{\ — p) < np and Cov^, £ 2 ) == n ( n — — P 2 )- Now 



P-P 



Hence 



< 



k(k - 1) 
rt 2 

k(k-l) 



2 

1 - - 

n 



fc-2 



fc-2 



= 1 



< 



fc-2 



fc 2 -fc 



nr 

F 
n 2 



n 

fc-i 



fc-i 



2 
n 

2 
n 



fe ~ 2 2fc 2 



Vax(iVi) < k 1 



fc-i 



2fc 2 



fc-2 



< 



n(log ?i + 6>)e~ a 2(log n + 6) 2 e 

n- 1 + ~ 
e~ e logn. 



2„-29 



n- 2 



Putting everything together shows 

Theorem 9. -Drop A: 6a/Zs independently and uniformly into n boxes. If k 
nlogn + On, and W denotes the number of empty boxes, then 



\L{W)-Poi e -e\\ TV <0 



( e 



Remark: The analysis above can be sharpened to give an error of order 
in theorem |5J See the remarks to example 2 in section 9.2. It is instructive to 
compare this bound with the results of theorem 6D in the book by Barbour, 
Hoist, and Janson. In the critical case where k — n\ogn + 9n, theorem 6D gives 
an explicit inequality which leads to an error of order as well. 

7. Multivariate Poisson Approximation 

This section gives useful bounds on the approximation of a vector of integer- 
valued random variables by a vector with independent Poisson coordinates. As 
an example, we treat the joint distribution of the number of fixed points and 
the number of times i goes to i + 1 in a random permutation. 
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Proposition 10. Let W = (Wx, • ■ ■ , Wd) be a random vector with values in N d 
and E(Wi) = A,; < oo. Let Z = (Z\, . . . , Z d ) have independent coordinates with 
Zi ~ Poi\ { . Let W' — (W{, . . . , W' d ) be defined on the same probability space as 
W with (W, W) an exchangeable pair. Then for all h : N d — > [0, 1], 



\Eh(W) -Eh(Z)\ < ya k 



fe=l 



E | A* - c k V w (A k )\ + E | Wk — c k V w (B k )\ 



(18) 



with a k = min(l, l-4A fc ), any choice of the {c k }, and 

A k = {W' k = W k + 1, Wj = W$ fork + l<j <d} 
Bk = {W k = W k -l,W j = W j fork + l<j<d}. 

Proof. Without loss of generality, assume that the Z's and the W's are defined 
on the same space and are independent. For h : N d — > [0, 1], define fi, fy, - ■ ■ , fd 
on N d as follows. Fix k G {1, . . . , d} and fix Wi,..., Wk-i,w k +i, ■ ■ ■ , Wd G N. 
Define fk(wi, . . . , w k -i, 0, w k +i, ■ ■ ■ ,w d ) = and 

hfk(u>i, ■ ■ ■ ,Wk-i,n,w k+1> . . . , w d ) = (n - l)f k (w 1 , . . .,w k -i,n- l,w k+1 , ...,w d ) 

+h(wi, . . . ,w k -i,n - 1, Wk+x, • • • , Wd)(l9) 
-Eh(wi, . . .,w k -i,Z k ,w k+ x, . . .,Wd)- 

Lemma Q shows that ||/fc||oo < min(l, 1.4Ai' 2 ). 

Now express the difference Eh(W) — Eh(Z) as a telescoping sum: 

d 

Eh(W)-Eh(Z) = J2i Eh (Zi, ■ • • , Z k - l: W k , . . . , W d )-Eh(Z 1: ...,Z k , W k+1 , W d )]. 
k=l 

(20) 

From the definition of f k given in (|19|) . 

Eh(Z u . . . , Z fc _ x , W fc , . . . , W d ) - Eh(Zi, . . . , Z k , W k+1 , ...,W d ) 

= E[\ k f k {Z u . . . , Z fc _ x , W fc + 1, . . . , W d ) - W k f k (Z x , . . . , Z k _ u W k ,..., W d )\. 

Now, for A k and B k as defined above, define 

T k = Cfc[/fc(Z 1; . . . , Z k _ u W k , . . . , W d )5 A * - f k (Z x , . . . , Zfc_ l5 W fel . . . , Wz)<5 Bfc ] 

for some c k , and note that by antisymmetry, E(Tfc) = 0. If 1 denotes the cr-field 
generated by Z\, . . . , Z d , W\, . . ■ , W d , then 

E(Tfc|J) = c k [f k (Z 1 ,...,Z k _ 1 ,W k + l,W k+l ,...,W d )F w (A k ) (21) 
-f k (Zi, Z k - U W k , W d )Pw{B k )}. 

As E(Tfc) = 0, the expression (|21|l can be inserted into the summand in (|20f) . 
yielding 



S. Chatterjee et al. /Exchangeable pairs and Poisson approximation 



87 



Eh(W) - Eh(Z) = E [( A ^ - c fc P(A fe ))A(^i, ■ • • , Z k _ u W k + 1, W k+U ...,m) 

k=l 

-{W k - c k P{B k ))f k (Z l7 Z fc _i, W k ,..., W d )]. 
From the claim fl%|l follows from lemma 0J. □ 

Example (d=2): Choose u uniformly in £„; let Wi = |{i : cr(i) = i}\ and 
W2 = \{i ■ cr(i) = i + 1}|. For W 2 , count cyclically, thus u(n) = 1 contributes to 
W%- It is intuitively clear that W\ and W2 have the same marginal distributions 
and that they are asymptotically independent. Proposition (|10|> allows a proof 
with error terms. Note that E(Wi) = E(W 2 ) = 1. 

Create an exchangeable pair (W[, W£) and (WijWb) by following a with a 
randomly chosen transposition. The sets A k , B k are 

A 1 = {W[ =Wi + l,W2 = W 2 } 

Bi = {W[ = Wi-l,W! 2 = W 2 } 

A 2 = {W 2 = W 2 + 1} 

B 2 = {W' 2 = W 2 -\\. 



Then 



2(n -W 1 -W 2 - W 3 ) 

1 FJ = ; n ■ 

n(n — 1) 



where 

VF 3 = \{i : <7 2 {i) = i, a{i) ^ i}U{i : a(i) = i+l,a(i+l) ^ i}U{i : a(i) = cr _1 (i)+l}|, 

n(n — 1J n(n — 1) 

where 

W4 = |{i : a(i) = i, a(i - 1) = i - 1}\. 

The calculations for P(A2|cr) and P(B 2 \a) are analogous to those carried out in 
section 0] Choosing c k — 2i ^- for k = 1,2 in proposition (fTUfl then yields the 
estimate 

\\L(W 1 ,W 2 )-L(Z 1 ,Z 2 )\\ TV < — , 

n 

where (Zi, Z 2 ) are independent Poisson random variables, each with mean 1. 
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8. Stein's Method for Poisson Process Approximation 
8. 1 . Introduction 

We begin with an example: consider, as in section a random permutation a 

{1 if = i, 

and let Wj — > X*; i.e., 
otherwise, r~f 

W is the number of matches up to time j. In the same way as in section 
one can show that Wj has an approximate Poisson distribution with parameter 

In this section we show that the point process Y t — W|±j for < t < 1 
converges to a Poisson process of rate 1. Similar results can be derived for the 
birthday problem: the number of birthday matches with [ty/n\ people converges 
to a Poisson process of rate 1. For the coupon-collector's problem, the number 
of new coupons collected from time n log n to time n log n + tn converges to a 
Poisson process of rate e~*. 

Recall that if ^ is a complete separable metric space and [i is a measure on ^ 
which is finite on bounded sets, a Poisson process of rate /i is a random discrete 
measure on y with the following properties: 

• The number of points N(A) in a bounded set A has a Poisson distribution 
with parameter (J-(A). 

• If Ai, . . . , Af. are disjoint bounded sets, then {N (Ai)}^ =1 are independent. 

Useful introductions to Poisson processes are found in Kingman {36} and 
Resnick 03]. The first use of Stein's method in Poisson process approximation 
was in 1988 in the paper jS] by Barbour, and the last chapter of Barbour, 
Hoist, and Janson jH| has a wonderful development of Stein's method for Poisson 
process approximation, using dependency graphs and the coupling approach. We 
show here how the method of exchangeable pairs can be used. Again, there are 
other approaches to proving Poisson process approximation. We find the papers 
|3*^1 and 001 by Kurtz particularly helpful. For a connection with Palm theory, 
see [T5] . 

Returning to our example, once we have the Poisson process approximation 
in place, one has various 'off the shelf results, e.g. 

• If there are k matches, their location is distributed as k random points in 
[0,1]. 

• One has the limiting distribution of the minimum (or maximum) distance 
between two matching times (see Gilbert and Pollak Feller |2fi|). 

Furthermore, one can harness some standard machinery for transformations 
of Poisson processes. Perhaps the three most important such results are the 
following: Let a Poisson process of rate [i be given on y. 

• (Mapping) If T : y — > X is a proper map then the image process is 
Poisson on X with rate fi T . 
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• (Spraying) Let K(y, dx) be a stochastic kernel on ^ x ^x- Replace each 
point y in the y process by a point chosen from K(y, dx). The new process 

is Poisson with rate / K (y , dx) u(dy) , 

• (Thinning) Delete points in the ^ process with probability P(y). The 

new process is Poisson with rate / P(y)/i(dy). 

J A 

With these examples as our motivation, we turn to a rigorous account. Let 
3 be a finite index set, and let {Xi}i e j be binary indicators with some joint 
distribution and with PpQ = 1) = Pi- Let {Wijigj be independent Poisson 
random variables with parameters pi. We give a development which allows us 
to bound the distance 

\\L(Xi : i £ 3)-L(Wi : i £ 3)\\ TV 

between the processes. This depends on the construction of an exchangeable pair 
(X, X'). In the matching problem, X is constructed from a random permutation 
er and X' is constructed by making a random transposition in a. 



8.2. The basic set-up 



Let 3 be a finite set, and let £ = x i$i be a configuration on 3. Thus £ is a 

measure on putting mass Xi at i, with Xi £ N. In fact, in our applications we 
will typically consider the case Xi £ {0, 1}. Let X D be the bounded measurable 
functions on configurations. Thus if / : N — > R is given, then £ ^ X) f(^) x i = 
J fd£ is in X , as are max{« : Xi > 0} and J f(i,j)d!;d£ for / : N x N — > R. 

Let E Q be the Poisson expectation operator on X Q . In other words, consider a 
probability space (CI, A, P) and an induced probability on the space of configura- 
tions on 3 given by writing £ = X Xi(uj)5i where the Xi ~ Poi(pi). Let X be the 
space of bounded measurable functions on ft with corresponding expectation 
operator E. As in section [21 there will be assigning a random 

variable to a function on configurations: 




In the matching example, the probability space is S n with the uniform dis- 
tribution, and the association a < — ► £ a = ^2xi(cr)di gives rise to the map 
(3T(ct) = T(£ CT ) for T £ X D . The aim is to show that E/3 = E D . To do this we 
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introduce 

U" o < — X 




3 f ^X 

with T the Stein operator 

WO = E p*[M£ + - MO] + E [M£ - - HO] (23) 

and 2f the subset of elements h e X Q with T Q ft, bounded. As explained in 
Barbour, Hoist, and Janson, this T is the generator of an immigration and 
death process Z(t) on 3 where particles are born at site i at exponential rate 
Pi and die at rate one. This process has the Poisson process of rate {pi} as 
a stationary distribution. This fact (or a straight-forward direct computation) 
shows that E T = 0. 

Note that for \3\ = 1 the operator is slightly different from the usual Stein 
operator for a Poisson process. This T Q is a "second difference" operator. We use 
it because of already available bounds on its inverse, but also as an introduction 
to this line of development. The inverse can be given a stochastic representation 

/>oo 

U o f(0 = - [E(f(Z(t))\0-Kf]dt 
Jo 

where Z{t) is the immigration and death process started at £. Barbour, Hoist, 
and Janson (pg. 209) show that 

T U f(S) = /(0 - E f. 

They also show that if /(£) = 5a{£) for some set of configurations A, then 

sup{|£7 /(£ + Si) — U f(£)\ :ie3,^a configuration} < 1. 

As before, let J be the bounded anti-symmetric functions on Vt x fl, and 
construct T via a symmetric probability Q with margins P: 

Tf(u ) ) = E q (f(u ) ,w')\u). 

We may choose a to be any linear map a : $ a — > 5". Stein's identity is 

E(/i(0) = E o (h(0) + - l3T ]U h. 

To proceed, we need to make an intelligent choice of a, make explicit the 
difference Ta — (3T Q , and use this expression to bound the remainder term. 
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Because of the different form of the Stein operator, the choice of a which seems 
most useful is: 

af(w, J) = c[/3/(&,) - 0f{&M«>,<»') - c\Pf(&) ~ 0f{i*)W> w) 
where 

uj ) 



1 if £ w ' = £w + 5j for some i, 
otherwise. 



Thus, 

pr h(u>) = x>[M&+<fc)-MU]-I>i[M& 

Tah(u>) = cY^[h{S u +6i)-h{S u )]Q(&=S u +5 i \u) 



i£j 



In our application, h = U a f, thus /i has bounded differences. It then follows 
that if c and the probability Q can be chosen so that 

|cQ(£w' = £u> + <M w) - pi | < e; and 
|cQ(^' =iu-Si\u)-Xi\ < 4 

then the error in the Poisson approximation is bounded by ^^(ej + e^)- 

iGJ 

Of course, in any application, some work is required. Here is a full version of 
the matching problem. 

Theorem 11. Let Xi(a), 1 < i < n indicate whether or not i is a fixed point 
of the random permutation a. Let Y{ be independent Poisson random variables, 
each with mean (i). Then 

4 

\\aXi : 1 < % < n) - UYi : 1 < % < n)\\ T v < -■ 

n 

Proof. It is easy to see that 

=tu + Oi\o-) = — -. TT— 

n(n — 1) 

and that 

<U>(£> = - <>i\ a) = — 

n(n — 1) 

with Zi(a) indicates whether a(a(i)) is i or not, and W = ^Xj. Choose c = 
^yi, and note that Pi = 1 for each i. We thus have that 



r l N 1 

= €v+Oi\a) 

n 



Zi(a) 
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and that 

m^=^-h\a)-X i {a)\ = Xi{a) ^\ 
It follows that if A is any set of configurations, 

|P(£(<r) G A)-E (S A )\ < -E(W + 2a 2 (a)) + -E(W 2 ) 

n n 

4 
n 

□ 



Remark: We have not treated the closely related area of compound Poisson 
approximation; see Erhardsson J2J and the references therein. 

9. The coupling approach to Poisson approximation 

Stein's method for Poisson approximation has been very actively developed 
along two lines, both quite different from the present account. The connections 
between the various approaches are not clear. In this section we give a brief 
development of the coupling method. In section^|we give a brief development 
of the dependency graph method. The book-length account of Barbour, Hoist, 
and Janson treats both coupling and dependency graphs in detail. The main 
reasons for including brief descriptions here are twofold: (a) as illustrated above, 
the many analytic tools developed by previous workers can easily be adapted 
for use with the present method of exchangeable pairs; (b) while the three 
approaches are similar, there are also real differences. Sometimes one or another 
approach is easier to apply or yields better bounds. 

The basics of the coupling method are laid out in section 19.11 Section 19.21 
treats our familiar examples: Poisson-binomial trials, the birthday problem, the 
coupon-collector's problem, and the matching problem. Section 19.31 illustrates 
one of the triumphs of the coupling approach: effortless bounds for negatively 
dependent summands. Our development leans heavily on lectures of Charles 
Stein. 



9.1. The basics 

From sectional we know that a random variable W has a Poisson distribution 
with parameter A if and only if, for each bounded measureable function /, 

E(Wf(W)) = XE(f(W + l)). 

This suggests that one approach to the question of whether W is approximately 
Poisson-distributed is to compare E(Wf(W)) and \E(f(W + 1)). The following 
simple lemma motivates a slightly mysterious construction to come. It appears 
in Stein [ET]. 
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Lemma 12. Let X = {Xi}i e j be {0, l}-valued random variables indexed by 
a finite set 3. Let A = VjE(JQ) and let I be uniformly distributed on 3 and 

independent of X . Then, for any bounded f, 

E(Wf(W))=XE(f(W)\X I = l). 

Proof. 

E(Wf(W)) = \3\ ■E{E(X I \X)f(W)} 
= \3\-E{X!f(W)} 
= \3\-E{X I E(f(W)\X I = l)} 
= XE(f(W)\X I = l). 

□ 

The lemma suggests comparing W+ 1 and a random variable W* having the 
distribution of W, given that the randomly chosen coordinate Xj is one. Many 
examples in which W* is constructed explicitly appear below. The following 
result gives a concrete bound on the total variation distance between the law of 
W and Poi\ in terms of the difference between W + 1 and W* . 

Proposition 13. Let X , I, and W be defined as in lemma l^lfy) . Let W* be 

defined on the same probability space with 

P(W* = w) = P(W = = 1). 

Then 

\\&(W) - PoixWrv < (1 - e-*)E\W + 1-W*\. 
Proof. Given A C N U {0}, let g(j) = S A (j) - Poi x (A) and set / = U a g, for U 
as defined in J5|. Then, from lemma (|12|l and equation ©, 

= E(Wf(W)-\f(W*)) 

= E(Wf(W) - Xf(W + 1) + Xf(W + 1) - Xf(W*)) 
= F(W G A) - Poix(A) + XE(f(W + 1) - f{W*)). 

Now, for any integers x < y, 

\m~f(y)\ < \f(x)-f(x + l)\ + \f(x+l)-f(x + 2)\ + ... + \f(y-l)-f( y )\, 

(1 - e~ A ) 

and from lemma (JJJ, \f(i + 1) — f(i)\ < , thus 

A 



\F(W G A) - Poi x {A)\ < (1 - e- A )E|T4^ + 1-W 



□ 



From these pieces, we get a bound for Poisson approximation by constructing 
a coupling (W, W*) with W* close toW+1. Stein [HI] and Barbour, Hoist, and 
Janson give many examples leading to good bounds in a host of problems. 

We next give a brief treatment of the basic probability problems treated in 
previous sections. 
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9. 2. Examples 

Example (Poisson-binomial trials): Let Xi, 1 < i < n be independent 

n 

with P(X l = l)= Pi = l- P(X l = 0). Let A = J^Pi and letW = X,. Then 



\L(W) - Poi x \\ TV < (1 6 } £p?. 



i=l 



Proof. We construct VF* 'backwards'. Pick X from the product measure, and 
independently pick I* with P(I* = i) = Set = 1 and X* = X l for 
7* . We claim that 

P(7* = i,X* = x) = P(I = i,X = x\Xi = 1). 

Indeed, if Xi — 0, then both sides are 0. If Xi = 1, then 

p(r = i,x* = x) = ^-]Jpf(i-p j y-^ 



and 



p(/ = i,x = x\ Xl = i) = = ^ n ** - *ti . 

v P(Xj = 1) A 

Note that the claim shows F{W* = j) = P(W = j\Xi = 1). To complete the 

2 

proof, observe that W + I — W* — Xi* , and that F(Xi* = 1) = ^ y- □ 
Remarks: 

1. Observe that the bound of this example is exactly the same as the bound 
given by the method of exchangeable pairs. 

2. The backwards construction of this example seems magical when you first 
encounter it. The method of exhangeable pairs seems much more straight- 
forward (at least to us). One benefit of the coupling approach is the clean 
bound of proposition JT3J|. 



Example 1 (The matching problem): Here, let Xi = 8in(i) with 7r a 

n 

randomly chosen permutation in S„ - Then W — Xi is the number of fixed 
points in 7r. Construct W* by picking / uniformly in {1, . . . , n} and then setting 

. = if7T(I)=I, 

1 {I J)vr if tt(I) = J^I. 
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Then if a 2 is the number of transpositions in 7r, 

{1 with probability 
-1 with probability 2aa, 
otherwise. 



95 



Then 



thus 



¥.\W + l-W*\ = EE^VK + 1-W*| 
= E 



2a 2 

— + — 

n n 



2 

n 



\\Z(W) - PohWrv < 



(l-e" 1 )2 



Example 2 (The coupon-collector's problem): Here k balls are dropped 
into n boxes and Xi is one or zero as box i is empty or not. Then W = Xi is 

zero if all of the boxes are covered. To construct a coupling, pick a box uniformly 
and simply distribute each ball in it at random into one of the other boxes. We 
claim that 

E(| W + l-^|)<(l-i) fe (l + ^. 

To prove this, let A be the number of balls in the box selected. Observe that 
W + 1 - W* > and that 

• If A = 0, then W + 1 - W* = 1. 

• If A > 0, then W + 1 - W* is the number of the A balls that fall into 
empty boxes. 

From this, 

E ( | W+ i-Tn)<m + w. 

n n 

Now, E(lT r ) = n (l — ^) k . To compute E(AW), observe that by symmetry this 
is K(AiW), with Ai the number of balls in box 1. Let Xi indicate whether box 

n 

i is empty, thus Xi = W, and let Yj indicate whether ball j falls into the 

i=l 
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first box, thus Yj = A%. Then 

i=i 

E(AiW) = E(AiXi) + (n - l)E(AiZ 2 ) 
= (n - l)fcE(YiX 2 ) 



Combining these bounds proves the claim. 

Remark: If k = n log n + cn for c fixed and n large, then proposition l|13fl 
shows that the number of empty boxes is well approximated by a Poisson dis- 
tribution with error of order i2S_™, This is better than the bound we obtained 

n 

in section fusing exchangeable pairs. In preliminary work we have shown that 
if the proof of section [S] is carried through by conditioning only on W, we get 
an error term of order as well. 



Example 3 (The birthday problem): Here k balls are dropped uniformly 
into n boxes. Let Xij indicate whether balls i and j fall into the same box. Let 
W = . To form W* , pick a random pair of indices I < J and move ball 

i<j 

J into the same box as ball /. 

• If balls / and J are in the same box already (call this event S), then 
\W + 1-W*\ = 1. 

• If ball / is in a box containing a balls and ball J is in a (different) box 
containing b balls, then \W + 1 — W* \ = \a— b\. 

Thus 

E(\W + I - W*\) = P{S) + E (\W + 1 - W*\ S c ^j - P(S C ). 
Now P(5) =^>(S|VK = w)P{W = w) = ^2-^-F(W = w) = -^rE{W) = -■ 
ForE/W + l-VK* ' 



S°j we may, by symmetry, consider the absolute difference 

between the number of balls in the first two boxes, given that each contains at 
least one ball. This is just the absolute difference between the number of balls 
in the first and second boxes when k — 2 balls are dropped into n boxes. By 
bounding this by twice the expected number of balls in the first box, namely 
— , and bounding P(5 C ) by 1, we get 

1 4- 2k 

E(\W + 1-W*\) < — — . 
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Remark: If k — dyfn with 9 fixed and n large, then proposition (|f 3|l shows 
that the number of matching pairs has a Poisson distribution of mean ^- with 
error of order -4= . 

9.3. Automatic coupling 

One of the many contributions of the book-length treatment on Poisson approx- 
imation by Barbour, Hoist, and Janson (1992) is a soft theory which leads to 
very neat result. Recall that a collection {X{\i & j of random variables is called 
negatively associated if for all disjoint A, B C 3 and monotone functions /, g, 

E(f(Xi;ie A)g(X f ,j G B)) < E(/(X i5 i G A))E(g(X r ,j G 5)). 

Here, / : M. m — > K is monotone if x ^ y implies /(x) < f(y) where x di y is the 
coordinate order. 

Two leading tools are: (a)If {Yi} are independent with log-concave densi- 
ties, and Zi are distributed as Yi given Y\ = k, then {Zi} are negatively 
associated. (b)Increasing functions of disjoint subsets of negatively associated 
variables are negatively associated. 

By way of example, if Yi are independent, with Yi ~ Poi Pi \, where = 1 

i 

and ATj have the law of Yi given Yi = k, then iV^ have the law of k balls 

i 

dropped into boxes via a multinomial allocation. They are negatively associated. 
If Xi is one or zero as iVj is bounded above by m or not (for fixed m > 0), then 
Xj are negatively associated. 

In Bosc-Einstein allocation, k balls are dropped into n boxes so that all con- 
figurations are equally likely. If Yi are independent geometric random variables 
with P(Yi = j) = qp> for < j < oo, then the distribution of Yi given Yi = k 

i 

has a Bose-Einstein distribution. These variables are thus negatively associated. 
Many further examples and tools are given in [8|. 

The motivation for this set-up is the following elegant theorem of Barbour, 
Hoist, and Janson. 

Theorem 14. Let {Xi}i e j be binary, negatively associated random variables. 
Let W = Xi have mean A and variance a 2 . Then 
ie 3 



Remarks: 



\\L(W) - Poi x \\ TV < (1 - e~ A ) ( 1 - — 



1. Of course, the mean equals the variance for a Poisson random variable. 
The theorem shows that if the mean is close to the variance, then the law 
is close to Poisson. 
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2. Mixtures of negatively associated variables are negatively associated. Dia- 
conis and Holmes [22] apply these ideas to study the birthday and coupon- 
collector's problems when balls are dropped into boxes with probabilities 
{pi} and a prior is put on pi. 

3. For variables presented as conditional given a sum, a host of analytic 
techniques are available for higher order expansions and large deviations 

H3- 



10. The dependency graph approach 

In this section we give a brief description of the dependency graph approach to 
Stein's method. This method is often useful when a natural local dependence 
structure exists; in these cases, this approach may be easier to apply than con- 
structing an exchangeable pair or a coupling. Following a description of the 
approach, an example of using a dependency graph to solve a variation of the 
birthday problem is given. 

Let be a set of binary random variables, with P(Xj = 1) = pi. A 

dependency graph for {X{\ is a graph with vertex set I = {1, . . . , n} and edge 
set E, such that if I\, 1% are disjoint subsets of I with no edges connecting them, 
then {Xi}i € i 1 and {Xi}i^i 2 are independent. Let Ni denote the neighborhood 
of i in the graph; that is, 

Ni = {j&I:(i,j)eE}U{i}. 

This framework yields the following bound, first proved in Qj : 

Theorem 15. Let {Xi}i e i be a finite collection of binary random variables with 
dependency graph (7, E); suppose P(Xj = 1) = Pi and P(X; = 1, Xj = 1) = pij . 
Let X = ^2,Pi and W = ^2 Xi. Then 



\\&(W) - PoixWrv < min(l, A" 1 ) 



X! J2 ^" + EE /'</'/ 

iei j£Ni\{i} i£l iSJVj 



Proof. Let A C N, and let / = U q 8a for U as in sectional Let Z ~ Poi\. Then 
F(Z e A) - ¥(W e A) = E[Wf(W)-Xf(W + l)} (24) 

= ^Epsq/cwo-pi/xw + i)]. 

iei 

Let Wi = W — Xi, and Vi — Xj . Note that Xi and Vi are independent and 
that 

X i f(W)=X i f(W i + l). 
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So from (j2U, 

F(Z e A) - ¥(W e A) = ^E[(Xi-pi)f(Wi + l)]+PiE\f(Wi + l)-f(W+l)] 

= t E [( X i-Pi)(f(Wi + i) - f(yi + 1))] +p.Mf(W l + 1) - f(W + l)}] . 

By lemma ©, \f{W t + 1) - /(W + 1)| < min(l, A" 1 ), thus 

\piElfiWi + 1) - f(W + 1)]| < min(l, A" 1 )^. 

(The second factor of occurs as P(W, 7^ VF) = p^.) Further, writing /(Wj + 
1) — ./"(V^ + 1) as a sum of telescoping terms yields: 

f(Wi + 1) - f(Vi + 1) < min(l, A" 1 )(W i - Fi) = min(l, A" 1 ) I ( , 

\ieiViV / 

which gives 

\E[(Xi -pi)(f(Wi + 1) - /(Vi + 1))]| < min(l, A -1 ) £ (pa+PiPj). 

j£Ni\i 

This proves the theorem. □ 



Remark: Here we have insisted that Xt be independent with {X, }.,^^. The 
following theorem, stated and proved in Barbour, Hoist, and Janson (1992), is 
a more general result in which it is only required that X t must not depend too 
strongly on {Xj}jgNi- The proof is very similar to that of theorem i|15|) . 

Theorem 16. Let {Xi}i e i be a finite set of binary random variables with 
¥(Xi = 1) = pi, and let A = and W = Y, x i- For each 1 G L let l t 

and If be disjoint subsets of I so that (If,If,{i}) partitions the set I. Define 
the random variables 

Yi = 

Zi = Xj. 

||«C(W) - P«a||tv < min(l, A" 1 ) YM +PiEZ i +E(X i Z i )]+min(l, A" 1 ) ^ 

iei iei 

(25) 

where rji is any quantity satisfying 

lEpq^ + i))-^^ + i)\< V i 



for every g : N U {0} 



[0,1]. 
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Remarks: 

1. The idea is to choose the sets If and If to be the indexes of those Xj 
on which Xi depends 'weakly' and 'strongly', respectively. If the Xi are 
equipped with a dependency graph (I, E) as is theorem (tT5ll , then using 
If — I \ Ni and I? = Ni\i recovers theorem <|15[) from theorem (|16fl ■ 

2. If If is well-chosen as those Xj on which Xi only depends weakly, then 
one potentially good choice of T)i is 

r h =E\E(X i \{X j ,jeIf})- Pi \. 

Example (A generalized birthday problem): Let [n] = {1, ...,n}, fix 
k > 2, and let / = {a C [n] : \a\ — k}. Color the points of [n] independently, 
choosing the color of each uniformly from a set of c colors. Let 

{1 if all points of a have the same color, 
otherwise, 

and let W = X a . Thus W > means that at least one group of k entries 

in [n] all have the same color. We show that W is approximately Poisson with 
parameter A = (^)c 1 ~ fe . 

To get the problem into the present framework, we need a dependency graph 
for the X a . The simplest choice is to observe that if a D j3 = 0, then X a and 
Xp are independent, thus one dependency graph is the one in which N a = {(3 : 
a n (3 7^ 0}. To apply theorem (JTSJ, note that for a C [n], 

Pa =F(X a = l) = c 1 - k 1 (26) 

and if \a n (3\ = I ^ 0, then 

p Q/3 =P(X ce = l,X^ = l) = c 1 -( 2fc -^. (27) 

Given a, a set [3 such that |a fl j3\ = £ can be chosen in („) ClZp) ways. Putting 
these observations into theorem l|15(l yields the following: 

Proposition 17. For positive integers c, k, and n, let W be the number of 
monochromatic k-tuples if an n-element set is colored with c colors, indepen- 
dently and uniformly. Let A = (^)c 1_fc . Then 

\\H(W)-Poi x \\ TV ^mintLA- 1 ) 



Remarks: 

1. Restricting to the cases k = 2, 3, the bound of proposition (|17|) is of the 
same order as the bounds obtained in section [SJ 



C)|e)(rJ)^*-' >+ (:)^|:C)(;::) 
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2. The dependency graph approach seems quite well suited to this problem; 
theorem (|17f) was obtained very easily and is rather more general than the 
results in section [5] while those arguments were rather technical. On the 
other hand, the generalized matching problem presented in section 0] was 
not so difficult via the method of exchangeable pairs, and the dependence 
structure is much less clear in that problem. Although there are problems 
for which both approaches work well (Poisson-binomial trials are an easy 
example; see Barbour, Hoist, and Janson for more involved examples), it 
seems that these two methods mostly are useful in different contexts. 

11. Some open problems 

Our treatment above has been expository and the classical problems studied 
were meant to help illuminate the basics of Stein's method of exchangeable 
pairs. A point is, this method may be used for more complex problems than the 
ones treated here. The following examples are all open as far as we know. 

11.1. Variants of the matching problem 

1. Permutations with restricted positions. Let A be a fixed nx n matrix with 

n 

zero-one entries. A permutation a S S n is compatible with A if a ia ^ = 

i=l 

1. If, for example, A is zero on the diagonal and one elsewhere, then those 
a which are compatible with A are the ones with no fixed points. The 
matching problem asks for the number of permutations with k violations. 
One also may investigate this problem for more general A. This is a rich 
subject; for pointers, one may look at the classical treatment in Riordan 
0Hj or the section on Rook polynomials in Stanley 0B| ■ Diaconis-Graham- 
Holmes |21| give a survey and applications in statistics. In addition to 
studying classical patterns such as the menage problem, one may ask for 
necessary and/or sufficient conditions on A to guarantee a Poisson limit. 

2. Other actions. Let G be a finite group and X a finite set with G acting on 
X. One may mimic the development in the example above by considering 
a matrix A indexed by X (i.e., with entries a xy ) with zero-one entries. 
Say g S G agrees with A if J^a^^ = 1. One could choose g uniformly 

X 

in G and ask for the distribution of violations. An example of statistical 
interest (Daniels's test - see has G the symmetric group S n and X 
is the set of ordered pairs {(i,j) ■ i ^ j} with g{i,j) — (g(i), g(j))- 

3. Non-uniform permutations. All of the development above has been for 
uniformly chosen permutations (or group elements). A nice problem is to 
develop Poisson approximation for natural families of non-uniform distri- 
butions. Here is a specific example: Diaconis and Critchlow JHj discuss 
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a variety of metrics on the permutation group. For example, one may take 
Spearman's p, which boils down to the squared differences ^(ct(z)-ct'(z)) 2 . 
Fixing a metric d, one can define a probability distribution on permuta- 
tions by 

Pg(a) = z - 1 0^o) i 0<9<l, 

where z is a normalizing constant and o~ a is the center of the distribution. 
These Mallows models have been widely applied; for extensive references, 
see Fligner and Verducci [27J or Diaconis and Ram [21]. 
The problem is this: for 8, <r , and d fixed, pick a from P# and look at the 
distribution of the number of fixed points. We conjecture that this will be 
Poisson under very mild conditions. To use Stein's method, an exchange- 
able pair must be formed. One simple way to do this is to pick a from Pg 
and then make a random transposition r. Use the Metropolis algorithm 
to get the second coordinate distribution to be Pg as well. Alternatively, 
the Gibbs sampler may be used. Of course, the problems outlined in the 
first two items above may be studied under these non-uniform models. 
Diaconis and Holmes [22] give some examples and further motivation. 

11.2. Variants of the coupon- collector's problem 

1. Non-uniform allocations. One can drop k balls into n boxes with the prob- 
ability of dropping a ball into box i equal to pi. This is studied by classical 
arguments (and without error terms) in Rosen |47|. Diaconis and Holmes 
|22] study Bose-Einstein allocation. They also mention the problem of 
putting a prior on (pi, . . . ,p n ) and studying the Bayes distribution. 

2. Complexes. The balls may be dropped in j at a time such that within 
each group of balls, all must land in different boxes. This actually arises 
in collecting baseball cards. Kolchin et al. [37] give some classical results. 

3. Coverage problems. A sweeping generalization occurs in the world of cov- 
erage processes. Let A be a compact metric space. Pick points x%, . . . ,Xk 
from a probability measure /i on A. For a fixed e > 0, what is the chance 
that the union of the e-balls about the xi cover A? Classical examples are 
dropping arcs on a circle (see Feller [21]) or caps on a sphere. Hall [HI] 
and Janson [32] give many results. Aldous has further literature. The 
widely studied k-SAT problem of computer science theory can be seen as a 
special case. Here, A is the hyper-cube Zj, k points are chosen randomly, 
and one wants to understand the chance that Hamming balls of a given 
radius cover the cube. 

11.3. The birthday problem 

Many natural variations have been studied: one may fix a graph, color the 
vertices with k colors chosen with probabilities pi for 1 < i < k, and ask how 
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many edges have the same color. The same kind of question may be asked 
for hyper-graphs. Some of these problems are treated by Aldous pQ, Arratia- 
Gordon-Goldstein or Barbour-Holst-Janson [S]. We mention here a lesser- 
known development. Camarri and Pitman |12j have shown that limits other 
than the Poisson can arise if non-uniform probabilities are used. The limiting 
measures that arise there are natural (related to the Mittag-Lcmcr function) and 
well worth further development. It is also natural to study how these limiting 
regimes occur in the graph variants discussed above. Developing Stein's method 
for these cases seems like a natural project. 

We close this brief section with a pointer to infinitely many other open prob- 
lems. David Aldous has written a remarkable book called Probability Approx- 
imations via the Poisson Clumping Heuristic. This is really a collection of in- 
sights and conjectures, almost all related to Poisson approximation. Almost all 
of hundreds of problems are open. 
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